Spam is generally defined as unwanted and unsolicited e-mail. Each day thousands of e-mail systems, such as Simple Mail Transfer Protocol (SMTP) mail systems, controlled by spammers, connect or attempt to connect to mail servers of large Internet Service Providers (ISPs) to transmit spam. The Internet Service providers may attempt to block e-mail from an Internet Protocol domain or address that is recognized as a spammer and is on a known blacklist. In addition, the Internet Service providers typically have spam filters, which attempt to eliminate or at least reduce, the amount of spam which gets through to user computers, or which is unintentionally classified as not spam.
Internet Service Providers first focus their efforts on either obtaining spam filtering rules from a vendor or developing their own from analysis of spam messages. Secondly, ISPs either obtain a spam Internet Protocol blacklist from a vendor, or they compile their own by analyzing “verdict results”, arriving at a reputation for each mail system or IP address sender and establishing thresholds to be used to determine whether a particular mail system's IP address should be added to their blacklist. “Verdict results” are determinations of whether an e-mail message is classified as spam or not spam by a spam filter. Internet Service Providers (ISPs) generally analyze all verdict results for all e-mail originating from a sending IP address to determine a reputation for that sending IP Address.
More specifically, the operation of the blocking and filtering process is as follows. When an originating mail system or IP address sender attempts to transmit email to a destination mail system, the originating IP address is first checked against a whitelist and then against a blacklist. If the IP address is on the whitelist, the connection and associated messages are accepted. Whitelists are compiled by analyzing historical data to identify trusted mail systems, by including mail system IP lists supplied by trusted companies, and by analyzing blocking complaints to identify trusted mail system. If a connection is accepted, the e-mail message is passed to a filtering process or spam filter to determine if it is spam. If the message is determined to be spam by the spam filter, then the message is either quarantined or deleted. If the message is determined not to be spam, the message is sent to the recipient's post office inbox on a mail server. E-mail that is identified as spam by a mail server of an Internet Service provider may be placed in a user's post office spam folder. A user can download e-mail from either a user's post office inbox or from a user's post office spam folder onto the user's computer or client computer.
If the originating IP address is on a blacklist, the connection and associated messages are rejected. An error is returned on rejected connections and, in many cases a non-delivery notice is sent back to the originator of a rejected message.
A problem with such prior solutions is that spammers can easily send spam that gets past the blacklists and spam filters. They send spam from a vast number of different IP addresses that have no reputation or at least not a bad reputation. They modify their spam messages as often as they need in order to get a sufficient amount of spam through the filters. They test their spam messages prior to initiating an attack to insure it is sufficient. As a result, ISPs are constantly updating their blacklists and their filtering rules after the fact with the hope that it may mitigate the next attack. Sometimes it takes hours to days to be able to identify and create new filters to catch the new spam and as a result most of the spam from a specific attack may get past the filters. It is delivered to the ISP's post office, waiting to be requested by the user to be either read by an online email client or delivered to their personal computer email client.
The effectiveness of the current filtering process is limited because it is very hard to mitigate attacks by simply filtering spam messages after the fact. Spammers easily change or randomize the content of the spam messages to by-pass even the best spam filters. Additionally, spammers can execute test spam attacks to determine whether their spam messages for a specific spam attack will get past the filtering defenses. Even if a high percent of the spam gets filtered out, the spammers will increase their volume until they get a sufficient amount of spam through the filters.
As a result of not being able to identify sufficient spam to meet thresholds that ISPs have set up to block malicious mail systems, many ISPs lower their blocking thresholds, which lowers their confidence with their blocking decisions. This can cause an increase in the blocking of legitimate mail systems or IP address senders and an increase in complaints to their care centers.
Over the last several years, a larger and larger proportion of the spam that is sent by spammers is getting by Internet Service Providers' defenses, and being delivered into user's post office inbox. It has gotten to the point where sometimes close to 100% of all spam received during a specific spam attack from a spammer is getting through the ISP's defenses. It is common for large ISPs to receive five hundred million spam messages each day, transmitted from tens of millions of unique Internet Protocol (IP) addresses, many of which have never before connected to the Internet Service Provider.
As a result of being unsure of the sender's identity, the lack of reputation, the ever increasing difficulty with ascertaining whether a message is spam, and the increasing effectiveness of spammers, ISPs have had a hard time improving the effectiveness of their current spam blocking and filtering processes. As a consequence, spam is increasingly being delivered to the ISP subscribers' post office inboxes and has significantly and adversely affected their experience with using e-mail. In addition, from an ISP's perspective, it has greatly affected the cost of providing service.
Spammers need to get their spam past the ISP blocking and spam filtering defenses and into many users' post office inboxes in order for users to consider purchasing their products. To obtain maximum selling potential, spammers have to send spam to millions of user's email addresses on a vast number of ISPs. Spam that has gotten past blocking and filtering defenses and deposited in a user's post office inbox is commonly called “missed spam”.
There are various devices known in the prior art for filtering e-mail and/or classifying e-mail as either spam or not spam. U.S. Pat. No. 7,219,148 to Routhwaite et. al. discloses a feedback loop for spam prevention. Users, known as “spam fighters” receive unfiltered e-mail messages and identify them as either spam or not spam. The feedback from the “spam fighters” is used to train improved spam filters.